Electronic apparatus, control method of electronic apparatus and computer readable recording medium

ABSTRACT

Provided herein is a control method of an electronic apparatus. The control method of an electronic apparatus includes: generating signal type information using an image data of a signal region portion of a signal lamp in a driving-related image data of a vehicle; and performing a driving-related guide of the vehicle using the generated signal type information.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from Korean Patent Application No.10-2014-0073041, filed on Jun. 16, 2014, in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein byreference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an electronic apparatus, a controlmethod of the electronic apparatus and a computer readable recordingmedium, and more particularly, to an electronic apparatus, a controlmethod of the electronic apparatus and a computer readable recordingmedium capable of recognizing a signal type of a signal lamp andperforming a vehicle driving related guide based on the recognition.

2. Description of the Related Art

The most important thing when a user drives a vehicle is a safe drivingand a prevention of any traffic accident. For this, a variety ofassistant devices and a safety device, for example, a safety belt, anairbag, etc. are provided in a vehicle in order to perform thefunctional functions of vehicle component devices.

Furthermore, a device, for example, a black box, is increasinglyinstalled in a vehicle so as to investigate the reasons for a vehicleaccident when a vehicle has an accident in such a manner to store therunning-related images of a vehicle and the data transferred fromvarious sensors. A portable terminal, for example, a smart phone, atablet, etc. is equipped with the function of a black box or anavigation application, etc. and is usually used as one of vehicledevices.

However, the applicability of such running-related images is actuallylow in the vehicle devices. More specifically, even though therunning-related images of a vehicle are obtained through a visionsensor, for example, a camera provided in the vehicle, the electronicapparatus of the vehicle is designed to simply display or transmit suchdata or generate a simple alarming information, for example, a linedeparting state, etc.

In addition, as an electronic apparatus for a vehicle which becomesnewly commercial in recent years, a HUD (Head Up Display) or anaugmented reality interface are proposed, however the applicability ofthe running-related images of a vehicle is a simple display or a simplegeneration of alarming information.

SUMMARY OF THE INVENTION

The present invention is made in an effort to resolve theabove-mentioned problems.

Accordingly, an object of the present invention is to provide anelectronic apparatus, a control method of the electronic apparatus and acomputer readable recording medium capable of generating signal typeinformation of a signal lamp using a driving related image data of avehicle and performing a vehicle driving related guide based on thegenerated signal type information.

To achieve the above objects, a control method of an electronicapparatus includes: generating signal type information using an imagedata of a signal region portion of a signal lamp in a driving-relatedimage data of a vehicle; and performing a driving-related guide of thevehicle using the generated signal type information.

In addition, the control method may further include: determining whetherthe vehicle is in a stopping state using the driving-related image data,wherein the generating of the signal type information is performed whenthe vehicle is in the stopping state.

In addition, the determining whether the vehicle is in the stoppingstate may include: generating a gray image data for the driving-relatedimage data; and determining whether the vehicle is in the stopping stateby comparing each of a plurality of frames included in the generatedgray image data.

In addition, the generating of the signal type information may include:determining a region-of-interest including the signal lamp in thedriving-related image data; and converting the image data of thedetermined region-of-interest based on a preset pixel value to generatea region-of-interest image data.

In addition, the generating of the signal type information may include:detecting the image data of the signal region portion of the signal lampby applying a first region having a first area and a second regionincluding the first region and having a second area to theregion-of-interest image data; comparing a difference between an areapixel value of the second region and an area pixel value of the firstregion with a preset area pixel value; and generating the signal typeinformation of the signal lamp as a result of the comparison.

In addition, the signal type information may be information foridentifying each of the plurality of signals which is displayed in thesignal lamp.

In addition, the performing of the driving-related guide of the vehiclemay include outputting a signal guide using the signal type information.

In addition, the performing of the driving-related guide of the vehiclemay include outputting a signal change guide using path information forroad guide of the vehicle and the signal type information.

In addition, the outputting may be performed if the vehicle is in astopping state during a preset time from timing when the signal of thesignal lamp is changed.

In addition, the outputting may include: generating an indicator forperforming the driving-related guide; and outputting the generatedindicator based on augmented reality.

Meanwhile, to achieve the above objects, an electronic apparatusincludes: a signal type information generation unit generating signaltype information using an image data of a signal region portion of asignal lamp in a driving-related image data of a vehicle; and a controlunit performing a driving-related guide of the vehicle using thegenerated signal type information.

In addition, the electronic apparatus may further include: a drivingstate determining unit determining whether the vehicle is in a stoppingstate using the driving-related image data, wherein the control unitcontrols the signal type information generation unit to generate thesignal type information if it is determined that the vehicle is in thestopping state.

In addition, the driving state determining unit may generate a grayimage data for the driving-related image data and compare each of theplurality of frames included in the generated gray image data todetermine whether the vehicle is in the stopping state.

In addition, the signal type information generation unit may determine aregion-of-interest including the signal lamp in the driving-relatedimage data; and convert the image data of the determinedregion-of-interest based on a preset pixel value to generate aregion-of-interest image data.

In addition, the signal type information generation unit may detect theimage data of the signal region portion of the signal lamp by applying afirst region having a first area and a second region including the firstregion and having a second area to the region-of-interest image data,compare a difference between an area pixel value of the second regionand an area pixel value of the first region with a preset area pixelvalue, and generate the signal type information of the signal lamp as aresult of the comparison.

In addition, the signal type information may be information foridentifying each of the plurality of signals which is displayed in thesignal lamp.

In addition, the control unit may control an output unit to output asignal guide using the signal type information.

In addition, the control unit may output a signal change guide usingpath information for road guide of the vehicle and the signal typeinformation.

In addition, the control unit may control the output unit to output thesignal change guide if the vehicle is in a stopping state during apreset time from timing when the signal of the signal lamp is changed.

In addition, the control unit may control the output unit to generatethe indicator for performing the driving-related guide and output thegenerated indicator based on the augmented reality.

Meanwhile, to achieve the above objects, a recording medium according toan embodiment of the present invention may have a program code forexecuting the above-described control method on a computer.

According to the above various embodiments of the present invention, itis possible to generate the signal type information of the signal lamplocated on a road where the vehicle stops or drives and perform thevehicle driving related guide based on the generated signal typeinformation.

Further, according to the above various embodiments of the presentinvention, it is possible to perform a driver assistance role by guidingthe signal to the driver of the vehicle using the signal typeinformation.

Further, according to the above various embodiments of the presentinvention, it is possible to promote the user convenience by accuratelyinforming the signal change using the navigation path and the signaltype information.

Further, according to the above various embodiments of the presentinvention, it is possible to provide the guide to the driver using themore intuitive method, by performing the vehicle driving related guidelike the road guide, the signal guide, the signal change guide, etc., onthe augmented reality.

BRIEF DESCRIPTION OF THE DRAWING

FIG. 1 is a block diagram depicting an electronic apparatus according toan embodiment of the present invention.

FIG. 2 is a view for describing a system network connected to theelectronic apparatus according to the embodiment of the presentinvention.

FIG. 3 is a flow chart depicting a method of generating the signal typeinformation of an electronic apparatus according to an embodiment of thepresent invention.

FIG. 4 is a flow chart depicting in detail a method of determining astopping state of a vehicle according to an embodiment of the presentinvention.

FIG. 5 is a view depicting a process of generating a gray image datafrom a driving-related image data according to an embodiment of thepresent invention.

FIG. 6 is a flow chart depicting a method of generating the signal typeinformation of an electronic apparatus according to an embodiment of thepresent invention.

FIG. 7 is a view depicting a process of generating a region-of-interestimage data from a driving-related image data according to an embodimentof the present invention.

FIG. 8 is a view depicting a region-of-interest image data correspondingto a stop signal, and a stop signal determining method according to anembodiment of the present invention.

FIG. 9 is a view depicting a region-of-interest image data correspondingto a straight signal, and a straight signal determining method accordingto an embodiment of the present invention.

FIG. 10 is a flow chart depicting a signal change guiding method of anelectronic apparatus according to an embodiment of the presentinvention.

FIG. 11 is a view depicting a signal change guide screen according to anexemplary embodiment of the present invention.

FIG. 12 is a view depicting an implementation form in case where acamera and an electronic apparatus are separate types according to anembodiment of the present invention.

FIG. 13 is a view depicting an implementation form in case where acamera and an electronic apparatus are integrated types according to anembodiment of the present invention.

FIG. 14 is a view depicting an implementation form which uses a HUD(Head-UP Display) and an electronic apparatus according to an embodimentof the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

The descriptions hereinafter propose the principles of the invention, soit is obvious that a person having ordinary skill in the art canimplement the principle of the invention even though it is not describedor depicted in the present specification and can invent variousapparatuses included in the concept and scope of the invention. Inaddition, it should be understood that the conditional terms andembodiments listed in the present invention are principally intended tohelp the concepts of the present invention to be fully understood, notlimiting the listed embodiments and states.

In addition, it should be understood that all the detailed descriptionslisting the principle, view point and embodiments as well as specificembodiments of the present invention are intended to include thestructural and functional equivalents of these matters. In addition, itshould be understood that these equivalent matters include all thedevices invented to perform the same functions irrespective of thecurrently known equivalent matters as well as the equivalent matters,namely, structures which will be developed in the future.

Therefore, for example, it should be understood that the block diagramsof the present specification are intended to show the conception viewpoints of the exemplary circuits which embody the principles of thepresent invention. In similar ways, all the flows, state conversiondiagrams, pseudo codes, etc. may be substantially expressed on acomputer readable medium and may represent various processes which canbe executed by a computer or a processor irrespective of whether thecomputer or the processor is clearly illustrated or not.

The functions of various element depicted in the drawings and includingthe processor or the functional blocks indicates in the form of similarconcepts may be executed using an exclusive hardware as well as thehardware which has abilities to execute the related software. When it isprovided by the processor, the above functions may provided by a singleexclusive processor, a single shared processor or a plurality ofindividual processors, and a part of them may be shared.

The correct use of the processor or the term which is suggested as aconcept similar therewith should not be interpreted in such a way toexclusively cite the hardware which has an ability to execute software,and it should be interpreted that it is indented to implicitly includeROM, RAM and a nonvolatile memory. Well known other hardware may beincluded.

In the claims of the present specification, the components expressed asa method of executing the functions recited in the detailed descriptionsare intended to include all the methods for executing the functionswhich include all types of software including a combination of circuitelements performing, for example, the above functions or afirmware/micro code, etc. and may be combined with an appropriatecircuit to execute the software. It should be understood that thepresent invention defined by such claims is combined with the functionsprovided by variously listed means and with the ways that the claimsrequire, so any means for providing the above functions should beunderstood to be equivalent to what can be recognized from the presentspecification.

The above-described objects, features and advantages could become clearwith the aid of the following descriptions in relation with theaccompanying drawings, and a person having ordinary skill in the art towhich the present invention pertains can easily implement the technicalconcepts of the present invention. In addition, while the presentinvention is being described, if it is determined that the descriptionswith respect to the known technology in relation with the presentinvention may make unclear the subject matters of the present invention,such detailed descriptions would be omitted.

Various embodiments of the present invention will be described withreference to the accompanying drawings.

FIG. 1 is a block diagram depicting an electronic apparatus according toan embodiment of the present invention. Referring to FIG. 1, theelectronic apparatus 100 may include all or a part of a storing unit110, an input unit 120, an output unit 130, a signal type informationgeneration unit 140, a driving state determining unit 150, an augmentedreality provision unit 160, a control unit 170, a communication unit180, and a sensing unit 190.

Here, the electronic apparatus 100 may be implemented using a smartphone, a tablet computer, a palm computer, a PDA (Personal DigitalAssistant), a PMP (Portable Multimedia Player), a smart glass, a projectglass, a navigation, a black box, etc. each of which may provide to avehicle driver a driving-related information.

Here, the driving state of the vehicle may include, not limited to, avariety of states where the vehicle is being driven by a driver, forexample, a stopping state of the vehicle, a running state of thevehicle, a parking state of the vehicle, etc.

The driving-related guide may include, not limited to, a variety ofguides to assist the driving of a vehicle driver, for example, a roadguide, a line departing guide, a forward vehicle starting guide, asignal guide, a signal change guide, a forward vehicle collision guide,a lane change guide, a lane guide, etc.

Here, the road guide may include, but limited to, an augmented realityroad guide performing road guide by combining various information, forexample, a user's position, direction, etc. with the images of theforward scenes of the currently driving vehicle, and a 2D(2-Dimensional) or 3D (3-Dimensional) road guide performing a road guideby combing the map data of the 2D or 3D with various information, forexample, a user's position, direction, etc. Here, the road guide may beinterpreted as an occasion wherein a driver drives on the vehicle aswell as a concept including a road guide where a user moves walking orrunning.

In addition, the line departing guide may be a guide to guide whetherthe running vehicle departs from the lane or not.

Further, the forward vehicle starting guide may guide whether anothervehicle in front of an own vehicle which is stopping start.

In addition, the signal guide may be a guide for guiding the signalstates of a signal lamp positioned in front of the driving vehicle, forexample, a stop signal, a straight signal, a left turn signal, a rightturn signal, etc. Here, the colors and types corresponding to eachsignal may be different for each nation. In case of Korea, the stopsignal is a red color, the straight signal is a green color, the leftturn signal is a green color and a left turn arrow, and the right turnsignal is a green color and a right turn arrow.

In addition, the signal change guide may be a guide for guiding that thesignal state, for example, a signal lamp positioned in front of thedriving vehicle, has changed. For an example, if the signal changed fromthe stop signal to the straight signal, this may be guided.

In addition, the forward vehicle collision prevention guide may be aguide for guiding a collision with the forward vehicle when the distanceto the vehicle which stops in front of the stopping or running vehiclebecomes within a predetermined distance.

In addition, the lane change guide may a guide for guiding a change fromthe lane where the vehicle is positioned to another lane for the sake ofa pathway guide to the destination.

In addition, the lane guide may be a guide for guiding the lane wherethe vehicle is currently positioned.

The driving-related images which help various driving guide functionsmay be photographed in real time by the camera hung in the forwarddirection. Here the camera may be a camera which may be integrallyformed with the electronic apparatus 100 hung in the vehicle so as tophotograph the forward scenes of the vehicle. In this case, the cameramay be integral with the smart phone, the navigation or the black box,and the electronic apparatus 100 may receive the images photographed bythe integral camera.

As another example, the camera may be hung different from the electronicapparatus 100 and may photograph the forward scenes of the vehicle. Inthis case, the camera may be a separate black box which hung for theforward direction of the vehicle, and the electronic apparatus 100 mayreceive the photographed images based on a wired/wireless communicationwith the separately hung black box or may receive the photographedimages when a storing medium for storing the photographed images of theblack box is inserted in the electronic apparatus 100.

Hereinafter, the electronic apparatus 100 according to an embodiment ofthe present invention will be described in more details based on theabove contents.

The storing unit 110 has a function for storing various data andapplications which are necessary for the operations of the electronicapparatus 100. In particular, the storing unit 110 may store the datanecessary for the operations of the electronic apparatus 100, forexample, OS, a pathway search application, a map data, etc. In addition,the storing unit 110 may store the data generated by the operations ofthe electronic apparatus 100, for example, a searched pathway data, areceived image, etc. In addition, the storing unit 110 may storeposition-related information of a plurality of signals included in thesignal lamp.

Here, the storing unit 110 may be implemented using a built-in typestoring element, for example, RAM (Random Access Memory), Flash Memory,ROM (Read Only Memory), EPROM (Erasable Programmable ROM) EEPROM(Electronically Erasable and Programmable ROM), a register, a hard disk,a removable disk, a memory card, USIM (Universal Subscriber IdentityModule), etc. or a detachable type storing element, for example, a USBmemory, etc.

The input unit 120 has a function for converting physical input from theoutside of the electronic apparatus 100 into a specific electricalsignal. Here, the input unit 120 may be all or part of a user input unit121 and a microphone unit 123.

The user input unit 121 may receive a user's input, for example, atouch, a push operation, etc. Here, the user input unit 120 may beimplemented using at least one among a various button types touch sensorwhich receive touch inputs, and a near sensor for receiving anapproaching motion.

The microphone unit 123 may receive a user's voice and sound from theinside or outside of the vehicle.

The output unit 130 is a device for outputting the data of theelectronic apparatus 100. Here, the output unit 130 may be all or partof a display unit 131 and an audio output unit 133.

The display unit 131 is a device for outputting data that the electronicapparatus 100 may visually recognize. The display unit 131 may beimplemented with a display unit provided at a front side of the housingof the electronic apparatus 100. Here, the display unit 131 is formedintegral with the electronic apparatus 100 and may output a visualrecognition data and is installed separate from the electronic apparatus100 like the HUD and may output a visual recognition data.

The audio output unit 133 is a device for outputting the data that theelectronic apparatus 100 may audibly recognize. The audio output unit133 may be formed of a speaker which may output in the forms of soundthe data which should be informed to the user of the electronicapparatus 100.

The communication unit 180 may provide to communicate with anotherdevice. The communication unit 180 may include, not limited to all orpart of a position data unit 181, a wireless internet unit 183, abroadcast transceiver unit 185, a mobile communication unit 186, a localarea communication unit 187 and a wired communication unit 189.

The position data unit 181 is a device for obtaining the position datathrough the GNSS (Global Navigation Satellite system). The GNSS means anavigation system which may calculate the position of the receiverterminal using radio signals from the satellite. As an example of theGNSS, there may, based on its operation body, be GPS (Global PositioningSystem), Galileo, GLONASS (Global Orbiting Navigational SatelliteSystem), COMPASS, IRNSS (Indian Regional Navigational Satellite System),QZSS (Quasi-Zenith Satellite System), etc. The position data unit 181 ofthe electronic apparatus 100 according to an embodiment of the presentinvention may obtain a position data by receiving a GNSS signal which isserved in a region where the electronic apparatus 100 is being used.

The wireless internet unit 183 is a device for connecting to thewireless internet, thus obtaining or transmitting the data. The wirelessinternet unit through the wireless internet unit 183 for connecting tothe wireless internet may include, but not limited to, WLAN (WirelessLAN), Wibro (Wireless broadband), Wimax (World interoperability formicrowave access), HSDPA (High Speed Downlink Packet Access), etc.

The broadcast transceiver unit 185 is a device for transmitting andreceiving broadcast signals through various broadcast systems. Thebroadcast system for transmitting and receiving through the broadcasttransceiver 185 may include, but not limited to, DMBT (DigitalMultimedia Broadcasting Terrestrial), DMBS (Digital MultimediaBroadcasting Satellite), MediaFLO (Media Forward Link Only), DVBH(Digital Video Broadcast Handheld), ISDBT (Integrated Services DigitalBroadcast Terrestrial), etc. The broadcast signal which may betransmitted or received through the broadcast transceiver unit 185 mayinclude, but not limited to, a traffic data, a life data, etc.

The mobile communication unit 186 may communicate by connecting to amobile communication network in compliance with various mobilecommunication criteria, for example, 3G (3rd Generation), 3GPP (3rdGeneration Partnership Project), LTE (Long Term Evolution), etc.

The local area communication unit 187 is a device for the sake of alocal area communication. The local area communication unit 187 mayprovide to communicate through Bluetooth, RFID (Radio FrequencyIdentification), infrared ray communication (IrDA, Infrared DataAssociation), UWB (Ultra WidBand), ZigBee, NFC (Near FieldCommunication), WiFi, etc.

The wired communication unit 189 is an interface device for connectingthe electronic apparatus 100 to another device through a wiredconnection. The wired communication unit 119 may be a USB module whichmay communicate through the USB port.

The communication unit 180 may communicate with another device using atleast one of the position data unit 181, a wireless internet unit 183, abroadcast transceiver unit 185, a mobile communication unit 186, a localarea communication unit 187, and a wired communication unit 189.

As an example, in case where the electronic apparatus 100 does notinclude a camera function, using at least one of the local areacommunication unit 187 and the wired communication unit 189, it ispossible to receive the images taken by the vehicle camera, for example,a black box, etc.

As another example, in case where a communication is made to multipledevices, any one of them communicate with the local area communicationunit 187, and the other one of them may communicate through the wiredcommunication unit 119.

The sensing unit 190 is a device for detecting the current state of theelectronic apparatus 100 and may include, but not limited to, all orpart of a motion sensing unit 191 and a light sensing unit 193.

The motion sensing unit 191 may detect the motion in the 3D space of theelectronic apparatus 100. The motion sensing unit 191 may be a 3-axisgeomagnetic sensor and a 3-axis acceleration sensor. The motion dataobtained by the motion sensing unit 191 is combined with the positiondata obtained by the position data unit 181, thus calculating a moreaccurate trace than the vehicle with the electronic apparatus 100.

The light sensing unit 193 is a device for measuring surroundingilluminance of the electronic apparatus 100 and allows to change thebrightness of the display unit 131 to the surrounding brightness usingthe illuminance data obtained through the light sensing unit 193.

The power unit 195 is a device for supplying power which is necessaryfor the operation of the electronic apparatus 100 and the operation ofanother device connected to the electronic apparatus 100. The power unit195 may be a device for receiving power from external power source, forexample, a battery in the electronic apparatus 100 or a vehicle. Inaddition, the power unit 195 may, depending on the type for receivingpower, be implemented in the form of a wired communication module 119 ora device for receiving power through a wireless connection.

Meanwhile, the control unit 170 controls the whole operations of theelectronic apparatus 100. In more detail, the control unit 170 maycontrol all or part of the storing unit 110, the input unit 120, theoutput unit 130, a signal type information generation unit 140, adriving state determining unit 150, the AR provision unit 160, thecommunication unit 180, and the sensing unit 190.

The signal type information generation unit 140 may generate signal typeinformation using an image data of a signal region portion of a signallamp in the driving-related image data.

In detail, the signal type information generation unit 140 may determinea region-of-interest including the signal lamp in the driving-relatedimage data and convert the determined region-of-interest image databased on a preset pixel value to generate the region-of-interest imagedata. Further, the signal type information generation unit 140 may applya first region having a first area and a second region including thefirst region and having a second area to the region-of-interest imagedata to detect the image data of the signal region portion of the signallamp. Further, signal type information generation unit 140 may compare adifference between an area pixel value of the second region and an areapixel value of the first region with a preset area pixel value andgenerate the signal type information of the signal lamp based on thecomparison result.

Here, the signal type information is information for identifying each ofthe plurality of signals which may be displayed on the signal lamp andmay include stop signal information, straight signal information, leftturn signal information, right turn signal, etc.

Meanwhile, the signal type information generation operation of theabove-mentioned signal type information generation unit 140 may beperformed when the driving state determining unit 150 determines thatthe vehicle is in the stopping state.

In detail, the driving state determining unit 150 may determine whetherthe vehicle is in the stopping state using the driving-related data. Inmore detail, the driving state determining unit 150 may generate thegray image data for the driving-related image data and sequentiallycompare each of the plurality of frames included in the generated grayimage data in a time order to determine whether the vehicle is in thestopping state.

As the determining result of the driving state determining unit 150, ifit is determined that the vehicle is in the stopping state, the controlunit 170 may control the signal type information generation unit 140 togenerate the signal type information.

Meanwhile, the control unit 170 may perform the driving-related guide ofthe vehicle using the signal type information generated from the signaltype information generation unit 140.

For example, the control unit 170 may control the output unit 130 tooutput the signal guide using the signal type information generated fromthe signal type information generation unit 140. In detail, the controlunit 170 may control the output unit 130 to output what the signal stateof the signal lamp is, for example, the stop signal, the straightsignal, the right turn, the left turn, etc., as an image or a voice.

As another example, the control unit 170 may control the output unit 130to output the signal change guide using the signal type information andthe path information for road guide which are generated from the signaltype information generation unit 140. In detail, the control unit 170may control the output unit 130 to output the signal change guide usingthe path information for road guide to a destination and the signal typeinformation when the electronic apparatus 100 performs the road guide tothe destination of the vehicle. That is, when the signal typeinformation generated from the signal type information generation unit140 is changed from the stop signal to the left turn signal in the statein which the path information of the vehicle stopping at an intersectionis in a straight direction, the control unit 170 may control the outputunit 130 not to output the signal change guide. Also, when the signaltype information generated from the signal type information generationunit 140 is changed from the stop signal to the straight signal in thestate in which the path information of the vehicle stopping at theintersection is in the straight direction, the control unit 170 maycontrol the output unit 130 to output the signal change guide. In thiscase, the control unit 170 may control the output unit 130 to output thesignal change guide if the vehicle is in the stopping state during apreset time from the signal change timing of the signal lamp.

Meanwhile, the control unit 170 may control the electronic apparatus 100to perform the driving-related guide based on the augmented reality.Here, the augmented reality may be a method of visually overlayingadditional information (for example, graphic element representing apoint of interest (POI), a graphic element representing a path to adestination, etc.) on a screen displaying a real world that a useractually sees and providing the overlaid additional information. In thiscase, the control unit 170 may be linked with the augmented realityprovision unit 160 to generate an indicator for performing thedriving-related guide and output the generated indicator through theoutput unit 130. For example, the augmented reality may be providedusing the HUD which uses the wind shield of the vehicle or an imageoverlay which uses a separate image output device. The augmented realityprovision unit 160 may generate a real image or an interface image, etc.which overlaps on the glass. Based on the above features, it is possibleto implement an augmented reality navigation or a vehicle infotainmentsystem.

FIG. 2 is a view for describing a system network which is connected toanother electronic apparatus according to an embodiment of the presentinvention. Referring to FIG. 2, the electronic apparatus 100 of anembodiment of the present invention may be implemented using variousdevices provided in the vehicle, for example, a navigation, a black box,a smart phone or other vehicle AR interface provision device and may beconnected with various communication networks and other electronicdevices 61 to 64.

In addition, the electronic apparatus 100 may calculate the currentposition and current timing in cooperation with a GPS module inaccordance with a radio signal from the satellite.

Each satellite 20 may transmit or receive L-band frequencies thefrequency bands of which are different. The electronic apparatus 100 maycalculate the current position based on the time which has lapsed untilthe L-band frequency from each satellite 20 reaches the electronicapparatus 100.

Meanwhile, the electronic apparatus 100 may connect to the network 30 ina wireless way through the communication unit 180 and through thecontrol station 40 (ACR) and the base station 50 (RAS). When theelectronic apparatus 100 gets connected to the network 30, it mayindirectly connected to the electronic devices 61 and 62, thusexchanging the data.

Meanwhile, the electronic apparatus 100 may indirectly connect to thenetwork 30 through another device 63 which has a communication function.For example, in case where the electronic apparatus 100 is not equippedwith a module which may connect to the network 30, it is possible tocommunicate with another device 63 which has a communication functionthrough the local area communication module.

FIG. 3 is a flow chart depicting a method of generating the signal typeinformation of an electronic apparatus according to an embodiment of thepresent invention. Referring to FIG. 3, first, the electronic apparatus100 may determine whether the vehicle is the stopping state using thedriving-related image data (S101).

Further, if it is determined that the vehicle is in the stopping state,the electronic apparatus 100 may generate the signal type informationusing the image data of the signal region portion of the signal lamp inthe driving-related image data of the vehicle (S102).

Further, the electronic apparatus 100 may perform the driving-relatedguide of the vehicle using the generated signal type information (S103).

FIG. 4 is a flow chart depicting in detail a method of determining astopping state of a vehicle according to an embodiment of the presentinvention. Referring to FIG. 4, first, the electronic apparatus 100 maygenerate the gray image data for the driving-related image data (S201).Here, the driving-related image of the vehicle may include images forthe stopping, running, etc., of the vehicle. Further, thedriving-related image of the vehicle is an image photographed by thecamera module included in the electronic apparatus 100 or may be animage that an image photographed by another apparatus is received by theelectronic apparatus 100. Further, the driving-related image of thevehicle may be a red green blue (RGB) (color) image. This, will bedescribed in detail with reference to FIG. 5.

FIG. 5 is a view depicting a process of generating a gray image datafrom a driving-related image data according to an embodiment of thepresent invention. Referring to FIG. 5, the driving state determiningunit 150 may receive the driving-related image data which is a colorimage 501 and perform gray conversion to generate the gray image data502.

Further, the electronic apparatus 100 may compare each of the pluralityof frames included in the generated gray image data (S202). In detail,the driving state determining unit 150 may sequentially compare each ofthe plurality of frames included in the generated gray image data in atime order.

Further, as the comparison result, the electronic apparatus 100 maydetermine whether the vehicle is in the stopping state (S203). As anexample, the driving state determining unit 150 may calculate thedifference between a current frame and a previous frame and compare thecalculated value with the preset value to determine whether the vehicleis in the stopping state.

According to a method of determining a stopping state of a vehicle asdepicted in FIG. 4, determining the stopping state of the vehicle basedon the gray image data, an image processing speed may be improved, andtherefore it is possible to quickly determine whether the vehicle is inthe stopping state in the driving state in which the running and thestopping of the vehicle are changed from hour to hour.

Meanwhile, the method of determining a stopping state of a vehicle asdepicted in FIG. 4 is only the embodiment of the present invention andtherefore is not limited thereto. Therefore, a method of determiningvarious stopping states of a vehicle like determining whether thevehicle is in the stopping state based on the sensing information of themotion sensing unit 191 may be used.

FIG. 6 is a flow chart depicting a method of generating the signal typeinformation of an electronic apparatus according to an embodiment of thepresent invention. Referring to FIG. 6, first, the electronic apparatus100 may determine the region-of-interest including the signal lamp inthe driving-related image data (S301).

In this case, the signal type information generation unit 140 maydetermine the region-of-interest including the signal lamp in thedriving-related image data using, for example, a vanishing point. Thatis, the signal type information generation unit 140 may extract a linefrom the image photographed by the camera while the vehicle is drivingand may extend the extracted line, and may determine a point where theextracted line crosses each other, as the vanishing point. Further, thesignal lamp is positioned at an upper region of the vanishing point andtherefore the signal type information generation unit 140 may determinethe determined upper region of the vanishing point as theregion-of-interest.

Meanwhile, as another example, the signal type information generationunit 140 may determine a predetermined region in the driving-relatedimage data as the region-of-interest.

Further, the electronic apparatus 100 may convert the image data of thedetermined region-of-interest based on the preset pixel value togenerate the region-of-interest image data (S302). In this case, thesignal type information generation unit 140 may, as an example, generatethe region-of-interest image data by adapting to the formula 1 the imagedata of the determined region of interest.

$\begin{matrix}{\frac{{x \times R} + {y \times G} + {z \times B}}{256} + K} & \lbrack {{Formula}\mspace{14mu} 1} \rbrack\end{matrix}$

Here R, G, B represent the R, G, B values of the image data of thedetermined region of interest, and K represents the previously set pixelvalue which is a criteria of the conversion, and x, y, x representspecific coefficients. For example, K may be 128 which is the averagepixel value.

Therefore, the signal type information generation unit 140 may generatethe region-of-interest image data which clearly distinguishes from thesignal region portion of the signal lamp.

Meanwhile, the color and/or forms corresponding to each of the multiplesignal included in the signal lamp may different from each nation. Incase of Korea, the stop signal is a red signal, a straight signal is agreen signal, the left turn signal is a green left arrow, and a rightturn signal is a green right arrow.

Therefore, the above previously set pixel values and the x, y, z mayhave different values depending on the algorithm.

Steps S301 and S302 will be described in detail with reference to FIG.7.

FIG. 7 is a view depicting a process of generating a region-of-interestimage data from a driving-related image data according to an embodimentof the present invention. Referring to FIG. 7, the signal typeinformation generation unit 140 may determine the region-of-interest 701including the signal lamp in the driving-related image data as depictedon the left side of FIG. 7.

Further, the signal type information generation unit 140 may convert theimage data of the determined region-of-interest 701 to generate aregion-of-interest image data 702 as depicted on the right side of FIG.7. In this case, the signal type information generation unit 140 mayconvert the image data of the determined region-of-interest based on thepreset pixel value to generate the region-of-interest image data 702.

Therefore, the signal type information generation unit 140 may generatethe region-of-interest in which a signal region portion 703 of a signallamp is clearly differentiated.

Meanwhile, the electronic apparatus 100 may detect the image data at aportion of a signal region of the signal lamp by applying the firstregion and the second region to the region-of-interest image data(S303). In more detail, the signal type information generation unit 140may detect the image data of the signal region portion of the signallamp by applying a first region having a first area to theregion-of-interest image data and a second region including the firstregion and having a second area. Here, the first and second regions maybe forms corresponding to the forms of the signal of the signal lamp. Asan example, in case where the signal of the signal lamp is circular,elliptical or quadrangular, the first and second regions may becircular, elliptical and quadrangular.

Meanwhile, the electronic apparatus 100 may compare the differencebetween the pixel value of the area of the second region and the pixelvalue of the area of the first region with a previously set area pixelvalue (S304). Here, the previously set area pixel values may change byreflecting the color and/or forms corresponding to the multiple signalsincluded in the signal lamp.

In addition, the electronic apparatus 100 may generate the signal typeinformation of the signal lamp based on a result of the comparison(S305).

The steps S303, S304, and S305 will be described in details withreference to FIGS. 8 and 9.

FIG. 8 is a view depicting a region-of-interest image data correspondingto a stop signal, and a stop signal determining method according to anembodiment of the present invention. In case where the signal lamp is ared stop signal, the signal type information generation unit 140converts the image data in the region of interest including the signallamp based on a previously set pixel value, as depicted on the left sideof FIG. 8, the region-of-interest image data 801 can be generated. Here,the red stop signal of the signal lamp can be generated like 802 basedon the region-of-interest image data 801.

In this case, the signal type information generation unit 140 may detectthe image data of the signal region portion 802 of the signal lamp byapplying the first region 804 and the second region 803 to theregion-of-interest image data 801.

In addition, the signal type information generation unit 140 comparesthe difference between the area pixel value of the second region 803 andthe area pixel value of the first region 804 with a previously set areapixel value, thus generating a signal type information of the signallamp. As an example, the signal type information generation unit 140 maydetermine the signal type information as a stop signal if the differencebetween the area pixel value of the second region 803 and the area pixelvalue of the first region 804 is smaller than the previously set areapixel value (namely, as depicted on the right side of FIG. 8, the areapixel value of the first region 804 is larger).

FIG. 9 is a view depicting a region-of-interest image data correspondingto a straight signal, and a straight signal determining method accordingto an embodiment of the present invention. In case where the signal lampis a green straight signal, the signal type information generation unit140 converts the image data in the region of interest including thesignal lamp based on a previously set pixel value, as depicted on theleft side of FIG. 9, the region-of-interest image data 901 may begenerated. Here, the green straight signal of the signal lamp may begenerated like 902 based on the region-of-interest image data 901.

In this case, the signal type information generation unit 140 may detectthe image data of the signal region portion 902 of the signal lamp byapplying the first region 904 and the second region 903 to theregion-of-interest image data 901.

In addition, the signal type information generation unit 140 maycompares the difference between the area pixel value of the secondregion 903 and the area pixel value of the first region 904 with apreviously set area pixel value, thus generating a signal typeinformation of the signal lamp. As an example, the signal typeinformation generation unit 140 may determine the signal typeinformation as a straight signal if the difference between the areapixel value of the second region 903 and the area pixel value of thefirst region 904 is larger than the previously set area pixel value(namely, as depicted on the right side of FIG. 9, the area pixel valueof the first region 904 is smaller).

Meanwhile, although not omitted in FIGS. 8 to 9, even the judgment ofthe turn signal like the left turn and the right turn may be similarlyperformed to the method. That is, the signal type information generationunit 140 may determine the signal type information using the methodand/or the shape (for example, arrow shape) of the image data of thesignal region portion of the signal lamp.

According to the method of generating the signal type information asdepicted in FIG. 6, the signal type information is determined as thestate in which the region-of-interest image data is averaged based onthe preset pixel value, and therefore the image processing speed may beimproved, such that the signal type information may be quicklydetermined in the state in which the running and the stopping of thevehicle are changed from hour to hour.

Meanwhile, the method of generating the signal type information asdepicted in FIG. 6 is only the embodiment of the present invention andtherefore is not limited thereto. Therefore, a method of generatingvarious signal type information such as determining the signal typeinformation, etc., by detecting the red signal corresponding to the stopsignal or detecting the green signal corresponding to the straightsignal from the driving-related image data which is the color image maybe used.

Meanwhile, according to the embodiment of the present invention, if itis determined that the signal type information is to be the stop signal,the signal type information generation unit 140 may previously positionthe first region and the second region in the region in which thestarting signal and the turn signal like the left turn/right turn in theregion-of-interest image data are displayed. In this case, the signaltype information generation unit 140 may position the first region andthe second region at the position in advance using the position relationinformation of the plurality of signals stored in the storing unit 110.In this case, it is possible to more quickly determine the change fromthe stop signal to the straight signal or the rotation signal like theleft turn/right turn by increasing the image processing speed.

FIG. 10 is a flow chart depicting a signal change guiding method of anelectronic apparatus according to an embodiment of the presentinvention. Referring to FIG. 10, the electronic apparatus 100 mayacquire the path information for road guide (S401). In detail, when theelectronic apparatus 100 performs the road guide to the destination ofthe vehicle, the control unit 170 may acquire the path information forroad guide at the current position of the vehicle.

Further, the electronic apparatus 100 may determine whether the signalis changed (S402). In detail, the control unit 170 may determine whetherthe signal is changed based on the signal type information generatedfrom the signal type information generation unit 140.

Further, when the signal is changed, the electronic apparatus 100 maydetermine whether the vehicle is running depending on the changed signal(S403). For example, when the signal type information generated from thesignal type information generation unit 140 is changed from the stopsignal to the left turn signal in the state in which the pathinformation of the vehicle stopping at the intersection is in thestraight direction, the control unit 170 may determine that the vehicleis not running. As another example, when the signal type informationgenerated from the signal type information generation unit 140 ischanged from the stop signal to the straight signal in the state inwhich the path information of the vehicle stopping at the intersectionis in the straight direction, the control unit 170 may determine thatthe vehicle is running. Further, when the vehicle is running, theelectronic apparatus 100 may determine whether the vehicle is in thestopping state during a preset time from the timing when the signal ofthe signal lamp is changed (S404).

Further, when the vehicle is in the stopping state for the preset time,the electronic apparatus 100 may output the signal change guide (S405).

FIG. 11 is a view depicting a signal change guide screen according to anexemplary embodiment of the present invention; Referring to FIG. 11, theelectronic apparatus 100 according to the embodiment of the presentinvention may display the signal change guide screen on the augmentedreality.

For example, when the signal state of the signal lamp is changed fromthe stop signal as depicted on the left side of FIG. 11 to the straightsignal as depicted on the right side of FIG. 11, the augmented realityprovision unit 160 may generate the overlaid indicator in the augmentedreality and output a generated signal change guide indicator 1101 on theaugmented reality as depicted on the right side of FIG. 11.

FIG. 12 is a view depicting an implementation form in case where acamera and an electronic apparatus are separate types according to anembodiment of the present invention. Referring to FIG. 12, a vehicleblack box 200 which is provided separate from the vehicle navigation 100may constitute the system according to an embodiment of the presentinvention using the wired/wireless communication method.

The vehicle navigation 100 may include, but not limited to a displayunit 145 provided at a front portion of a navigation housing 191, anavigation controlling key 193, and a navigation microphone 195.

The vehicle black box 200 can obtain a data of a vehicle during therunning or stop of the vehicle. Namely, it is possible to photograph theimages during the running of the vehicle and the images even when thevehicle is parked. The quality of the images obtained through thevehicle black box 200 may be constant or vary. As an example, thequality of the images before or after the occurrence of an accident maybe high, and in the normal occasions, the quality of the images is lowso as to minimize the required storing space since it needs to storeimportant images.

The vehicle black box 200 may include, but not limited to, a black boxcamera 222, a black box microphone 224 and an attaching unit 281.

Meanwhile, FIG. 12 depicts that the vehicle black box 200 providedseparate from the vehicle navigation 100 is connected in awired/wireless communication method, but the vehicle navigation 100 andthe vehicle black box 200 may not be connected in the wired/wirelesscommunication method. In this case, if a storing medium capable ofstoring the photographed images of the black box 200 is inserted in theelectronic apparatus 100, the electronic apparatus 200 may have afunction of the vehicle navigation 100 or the vehicle navigation 100 maybe integrally provided. This configuration, as depicted in FIG. 13, willbe described in detail.

FIG. 13 is a view depicting the implementation form wherein the cameraand the electronic apparatus are integral, according to an embodiment ofthe present invention. Referring to FIG. 13, if the electronic apparatushas a camera function, the user can install the electronic apparatuswhich allows a camera portion of the electronic apparatus to photographthe forward scenes of the vehicle and the display portion of theelectronic apparatus to recognize the user, thus implementing the systemaccording to an embodiment of the present invention.

FIG. 14 is a view depicting the implementation form by using the HUD(Head-UP Display) and the electronic apparatus according to anembodiment of the present invention. Referring to FIG. 14, theelectronic apparatus may display the AR guide screen on the head-updisplay with the help of the head-up display and the wired/wirelesscommunication.

Meanwhile, the control method of the electronic apparatus according tovarious embodiments of the present invention are implemented in the formof program codes, which may be provided to each server or device in astate where such program devices are stored in various non-transitorycomputer readable medium.

The non-transitory computer readable medium is not a medium, forexample, a register, a cash, a memory, etc., which is designed to storefor short time period, but a medium which can store datasemi-permanently and can be read by a device. In detail, the abovevarious applications or programs may be stored in the non-transitoryreadable medium, for example, CD, DVD, a hard disk, a blue ray disk,USB, a memory card, ROM, etc.

In addition, as the present invention may be embodied in several formswithout departing from the spirit or essential characteristics thereof,it should also be understood that the above-described examples are notlimited by any of the details of the foregoing description, unlessotherwise specified, but rather should be construed broadly within itsspirit and scope as defined in the appended claims, and therefore allchanges and modifications that fall within the meets and bounds of theclaims, or equivalences of such meets and bounds are therefore intendedto be embraced by the appended claims.

What is claimed is:
 1. A control method of an electronic apparatus,comprising: generating signal type information using an image data of asignal region portion of a signal lamp in a driving-related image dataof a vehicle; and performing a driving-related guide of the vehicleusing the generated signal type information.
 2. The control method ofclaim 1, further comprising: determining whether the vehicle is in astopping state using the driving-related image data, wherein thegenerating of the signal type information is performed when the vehicleis in the stopping state.
 3. The control method of claim 2, wherein thedetermining whether the vehicle is in the stopping state includes:generating a gray image data for the driving-related image data; anddetermining whether the vehicle is in the stopping state by comparingeach of a plurality of frames included in the generated gray image data.4. The control method of claim 2, wherein the generating of the signaltype information includes: determining a region-of-interest includingthe signal lamp in the driving-related image data; and converting theimage data of the determined region-of-interest based on a preset pixelvalue to generate a region-of-interest image data.
 5. The control methodof claim 4, wherein the generating of the signal type informationincludes: detecting the image data of the signal region portion of thesignal lamp by applying a first region having a first area and a secondregion including the first region and having a second area to theregion-of-interest image data; comparing a difference between an areapixel value of the second region and an area pixel value of the firstregion with a preset area pixel value; and generating the signal typeinformation of the signal lamp as a result of the comparison.
 6. Thecontrol method of claim 2, wherein the signal type information isinformation for identifying each of the plurality of signals which isdisplayed in the signal lamp.
 7. The control method of claim 6, whereinthe performing of the driving-related guide of the vehicle includesoutputting a signal guide using the signal type information.
 8. Thecontrol method of claim 6, wherein the performing of the driving-relatedguide of the vehicle includes outputting a signal change guide usingpath information for road guide of the vehicle and the signal typeinformation.
 9. The control method of claim 8, wherein the outputting isperformed if the vehicle is in a stopping state during a preset timefrom timing when the signal of the signal lamp is changed.
 10. Thecontrol method of claim 7, wherein the outputting includes: generatingan indicator for performing the driving-related guide; and outputtingthe generated indicator based on augmented reality.
 11. An electronicapparatus, comprising: a signal type information generation unitgenerating signal type information using an image data of a signalregion portion of a signal lamp in a driving-related image data of avehicle; and a control unit performing a driving-related guide of thevehicle using the generated signal type information.
 12. The electronicapparatus of claim 11, further comprising: a driving state determiningunit determining whether the vehicle is in a stopping state using thedriving-related image data, wherein the control unit controls the signaltype information generation unit to generate the signal type informationif it is determined that the vehicle is in the stopping state.
 13. Theelectronic apparatus of claim 12, wherein the driving state determiningunit generates a gray image data for the driving-related image data andcompares each of the plurality of frames included in the generated grayimage data to determine whether the vehicle is in the stopping state.14. The electronic apparatus of claim 12, wherein the signal typeinformation generation unit determines a region-of-interest includingthe signal lamp in the driving-related image data and converts the imagedata of the determined region-of-interest based on a preset pixel valueto generate a region-of-interest image data.
 15. The electronicapparatus of claim 14, wherein the signal type information generationunit detects the image data of the signal region portion of the signallamp by applying a first region having a first area and a second regionincluding the first region and having a second area to theregion-of-interest image data, compares a difference between an areapixel value of the second region and an area pixel value of the firstregion with a preset area pixel value, and generates the signal typeinformation of the signal lamp as a result of the comparison.
 16. Theelectronic apparatus of claim 12, wherein the signal type information isinformation for identifying each of the plurality of signals which isdisplayed in the signal lamp.
 17. The electronic apparatus of claim 16,wherein the control unit controls an output unit to output a signalguide using the signal type information.
 18. The electronic apparatus ofclaim 16, wherein the control unit outputs a signal change guide usingpath information for road guide of the vehicle and the signal typeinformation.
 19. The electronic apparatus of claim 18, wherein thecontrol unit controls the output unit to output the signal change guideif the vehicle is in a stopping state during a preset time from timingwhen the signal of the signal lamp is changed.
 20. A recording mediumrecorded with a program code for executing the control method of claim 1on a computer.